{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N7ITxKLUkX0v"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "yOYx6tzSnWQ3"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6xgB0Oz5eGSQ"
      },
      "source": [
        "# Introduction to graphs and functions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w4zzZVZtQb1w"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/guide/intro_to_graphs\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/guide/intro_to_graphs.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/guide/intro_to_graphs.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/guide/intro_to_graphs.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RBKqnXI9GOax"
      },
      "source": [
        "# Introduction to Graphs and `tf.function`\n",
        "\n",
        "This guide goes beneath the surface of TensorFlow and Keras to see how TensorFlow works.  If you instead want to immediately get started with Keras, please see [our collection of Keras guides](keras/).\n",
        "\n",
        "In this guide you'll see the core of how TensorFlow allows you to make simple changes to your code to get graphs, and how they are stored and represented, and how you can use them to accelerate and export your models.\n",
        "\n",
        "Note: For those of you who are only familiar with TensorFlow 1.x, this guide demonstrates a very different view of graphs.\n",
        "\n",
        "This is a short-form introduction; for a full introduction to these concepts, see [the `tf.function` guide](function).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v0DdlfacAdTZ"
      },
      "source": [
        "## What are graphs?\n",
        "\n",
        "In the previous three guides, you have seen TensorFlow running **eagerly**.  This means TensorFlow operations are executed by Python, operation by operation, and returning results back to Python. Eager TensorFlow takes advantage of GPUs, allowing you to place variables, tensors, and even operations on GPUs and TPUs.  It is also easy to debug.\n",
        "\n",
        "For some users, you may never need or want to leave Python.\n",
        "\n",
        "However, running TensorFlow op-by-op in Python prevents a host of accelerations otherwise available. If you can extract tensor computations from Python, you can make them into a *graph*.  \n",
        "\n",
        "**Graphs are data structures that contain a set of `tf.Operation` objects, which represent units of computation; and `tf.Tensor` objects, which represent the units of data that flow between operations.** They are defined in a `tf.Graph` context. Since these graphs are data structures, they can be saved, run, and restored all without the original Python code.\n",
        "\n",
        "This is what a simple two-layer graph looks like when visualized in TensorBoard.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FvQ5aBuRGT1o"
      },
      "source": [
        "![a two-layer tensorflow graph](\t\n",
        "https://storage.cloud.google.com/tensorflow.org/images/two-layer-network.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DHpY3avXGITP"
      },
      "source": [
        "## The benefits of graphs\n",
        "\n",
        "With a graph, you have a great deal of flexibility.  You can use your TensorFlow graph in environments that don't have a Python interpreter, like mobile applications, embedded devices, and backend servers.  TensorFlow uses graphs as the format for saved models when it exports them from Python.\n",
        "\n",
        "Graphs are also easily optimized, allowing the compiler to do transformations like:\n",
        "\n",
        "* Statically infer the value of tensors by folding constant nodes in your computation *(\"constant folding\")*.\n",
        "* Separate sub-parts of a computation that are independent and split them between threads or devices.\n",
        "* Simplify arithmetic operations by eliminating common subexpressions.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o1x1EOD9GjnB"
      },
      "source": [
        "There is an entire optimization system, [Grappler](./graph_optimization.ipynb), to perform this and other speedups.\n",
        "\n",
        "In short, graphs are extremely useful and let your TensorFlow run **fast**, run **in parallel**, and run efficiently **on multiple devices**.\n",
        "\n",
        "However, you still want to define our machine learning models (or other computations) in Python for convenience, and then automatically construct graphs when you need them."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pSZebVuWxDXu"
      },
      "source": [
        "# Tracing graphs\n",
        "\n",
        "The way you create a graph in TensorFlow is to use `tf.function`, either as a direct call or as a decorator."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "goZwOXp_xyQj"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import timeit\n",
        "from datetime import datetime"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HKbLeJ1y0Umi"
      },
      "outputs": [],
      "source": [
        "# Define a Python function\n",
        "def function_to_get_faster(x, y, b):\n",
        "  x = tf.matmul(x, y)\n",
        "  x = x + b\n",
        "  return x\n",
        "\n",
        "# Create a `Function` object that contains a graph\n",
        "a_function_that_uses_a_graph = tf.function(function_to_get_faster)\n",
        "\n",
        "# Make some tensors\n",
        "x1 = tf.constant([[1.0, 2.0]])\n",
        "y1 = tf.constant([[2.0], [3.0]])\n",
        "b1 = tf.constant(4.0)\n",
        "\n",
        "# It just works!\n",
        "a_function_that_uses_a_graph(x1, y1, b1).numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MT7U8ozok0gV"
      },
      "source": [
        "`tf.function`-ized functions are [Python callables]() that work the same as their Python equivalents.  They have a particular class (`python.eager.def_function.Function`), but to you they act just as the non-traced version.\n",
        "\n",
        "`tf.function` recursively traces any Python function it calls."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rpz08iLplm9F"
      },
      "outputs": [],
      "source": [
        "def inner_function(x, y, b):\n",
        "  x = tf.matmul(x, y)\n",
        "  x = x + b\n",
        "  return x\n",
        "\n",
        "# Use the decorator\n",
        "@tf.function\n",
        "def outer_function(x):\n",
        "  y = tf.constant([[2.0], [3.0]])\n",
        "  b = tf.constant(4.0)\n",
        "\n",
        "  return inner_function(x, y, b)\n",
        "\n",
        "# Note that the callable will create a graph that\n",
        "# includes inner_function() as well as outer_function()\n",
        "outer_function(tf.constant([[1.0, 2.0]])).numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P88fOr88qgCj"
      },
      "source": [
        "If you have used TensorFlow 1.x, you will notice that at no time did you need to define a `Placeholder` or `tf.Sesssion`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wfeKf0Nr1OEK"
      },
      "source": [
        "## Flow control and side effects\n",
        "\n",
        "Flow control and loops are converted to TensorFlow via `tf.autograph` by default.  Autograph uses a combination of methods, including standardizing loop constructs, unrolling, and [AST](https://docs.python.org/3/library/ast.html) manipulation.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PFObpff1BMEb"
      },
      "outputs": [],
      "source": [
        "def my_function(x):\n",
        "  if tf.reduce_sum(x) <= 1:\n",
        "    return x * x\n",
        "  else:\n",
        "    return x-1\n",
        "\n",
        "a_function = tf.function(my_function)\n",
        "\n",
        "print(\"First branch, with graph:\", a_function(tf.constant(1.0)).numpy())\n",
        "print(\"Second branch, with graph:\", a_function(tf.constant([5.0, 5.0])).numpy())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hO4DBUNZBMwQ"
      },
      "source": [
        "You can directly call the Autograph conversion to see how Python is converted into TensorFlow ops.  This is, mostly, unreadable, but you can see the transformation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8x6RAqza1UWf"
      },
      "outputs": [],
      "source": [
        "# Don't read the output too carefully.\n",
        "print(tf.autograph.to_code(my_function))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZ4Ieg6tBE6l"
      },
      "source": [
        "Autograph automatically converts `if-then` clauses, loops, `break`, `return`, `continue`, and more.\n",
        "\n",
        "Most of the time, Autograph will work without  special considerations.  However, there are some caveats, and the [tf.function guide](./function.ipynb) can help here, as well as the [complete autograph reference](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/index.md)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A6NHDp7vAKcJ"
      },
      "source": [
        "## Seeing the speed up\n",
        "\n",
        "Just wrapping a tensor-using function in `tf.function` does not automatically speed up your code.  For small functions called a few times on a single machine, the overhead of calling a graph or graph fragment may dominate runtime.  Also, if most of the computation was already happening on an accelerator, such as stacks of GPU-heavy convolutions, the graph speedup won't be large.\n",
        "\n",
        "For complicated computations, graphs can provide a significant speedup.  This is because graphs reduce the Python-to-device communication and perform some speedups.\n",
        "\n",
        "This code times a few runs on some small dense layers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zbNndv-0BeO4"
      },
      "outputs": [],
      "source": [
        "# Create an oveerride model to classify pictures\n",
        "class SequentialModel(tf.keras.Model):\n",
        "  def __init__(self, **kwargs):\n",
        "    super(SequentialModel, self).__init__(**kwargs)\n",
        "    self.flatten = tf.keras.layers.Flatten(input_shape=(28, 28))\n",
        "    self.dense_1 = tf.keras.layers.Dense(128, activation=\"relu\")\n",
        "    self.dropout = tf.keras.layers.Dropout(0.2)\n",
        "    self.dense_2 = tf.keras.layers.Dense(10)\n",
        "\n",
        "  def call(self, x):\n",
        "    x = self.flatten(x)\n",
        "    x = self.dense_1(x)\n",
        "    x = self.dropout(x)\n",
        "    x = self.dense_2(x)\n",
        "    return x\n",
        "\n",
        "input_data = tf.random.uniform([60, 28, 28])\n",
        "\n",
        "eager_model = SequentialModel()\n",
        "graph_model = tf.function(eager_model)\n",
        "\n",
        "print(\"Eager time:\", timeit.timeit(lambda: eager_model(input_data), number=10000))\n",
        "print(\"Graph time:\", timeit.timeit(lambda: graph_model(input_data), number=10000))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kNGuLnjK1c5U"
      },
      "source": [
        "### Polymorphic functions\n",
        "\n",
        "When you trace a function, you create a `Function` object that is **polymorphic**.  A polymorphic function is a Python callable that encapsulates several concrete function graphs behind one API.\n",
        "\n",
        "You can use this `Function` on all different kinds of `dtypes` and shapes.  Each time you invoke it with a new argument signature, the original function gets re-traced with the new arguments.  The `Function` then stores the `tf.Graph` corresponding to that trace in a `concrete_function`.  If the function has already been traced with that kind of argument, you just get your pre-traced graph.\n",
        "\n",
        "Conceptually, then:\n",
        "* A **`tf.Graph`** is the raw, portable data structure describing a computation\n",
        "* A **`Function`** is a caching, tracing, dispatcher over ConcreteFunctions\n",
        "* A **`ConcreteFunction`** is an eager-compatible wrapper around a graph that lets you execute the graph from Python\n",
        "\n",
        "### Inspecting polymorphic functions\n",
        "\n",
        "You can inspect `a_function`, which is the result of calling `tf.function` on the Python function `my_function`.  In this example, calling `a_function` with three kinds of arguments results in three different concrete functions.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7heuYuwn2edE"
      },
      "outputs": [],
      "source": [
        "print(a_function)\n",
        "\n",
        "print(\"Calling a `Function`:\")\n",
        "print(\"Int:\", a_function(tf.constant(2)))\n",
        "print(\"Float:\", a_function(tf.constant(2.0)))\n",
        "print(\"Rank-1 tensor of floats\", a_function(tf.constant([2.0, 2.0, 2.0])))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s1c8db0cCW2k"
      },
      "outputs": [],
      "source": [
        "# Get the concrete function that works on floats\n",
        "print(\"Inspecting concrete functions\")\n",
        "print(\"Concrete function for float:\")\n",
        "print(a_function.get_concrete_function(tf.TensorSpec(shape=[], dtype=tf.float32)))\n",
        "print(\"Concrete function for tensor of floats:\")\n",
        "print(a_function.get_concrete_function(tf.constant([2.0, 2.0, 2.0])))\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JLTNuv_CCZXK"
      },
      "outputs": [],
      "source": [
        "# Concrete functions are callable\n",
        "# Note: You won't normally do this, but instead just call the containing `Function`\n",
        "cf = a_function.get_concrete_function(tf.constant(2))\n",
        "print(\"Directly calling a concrete function:\", cf(tf.constant(2)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PTHNiHLXH9es"
      },
      "source": [
        "In this example, you are seeing pretty far into the stack.  Unless you are specifically managing tracing, you will not normally need to call concrete functions directly as shown here."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V11zkxU22XeD"
      },
      "source": [
        "# Reverting to eager execution\n",
        "\n",
        "You may find yourself looking at long stack traces, specially ones that refer to `tf.Graph` or `with tf.Graph().as_default()`.  This means you are likely running in a graph context.  Core functions in TensorFlow use graph contexts, such as Keras's `model.fit()`.\n",
        "\n",
        "It is often much easier to debug eager execution.  Stack traces should be relatively short and easy to comprehend.\n",
        "\n",
        "In situations where the graph makes debugging tricky, you can revert to using eager execution to debug. \n",
        "\n",
        "Here are ways you can make sure you are running eagerly:\n",
        "\n",
        "* Call models and layers directly as callables\n",
        "\n",
        "* When using Keras compile/fit, at compile time use **`model.compile(run_eagerly=True)`**\n",
        "\n",
        "* Set global execution mode via  **`tf.config.run_functions_eagerly(True)`**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iTHvdQfRICJb"
      },
      "source": [
        "### Using `run_eagerly=True`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kqzBV2rSzvpC"
      },
      "outputs": [],
      "source": [
        "# Define an identity layer with an eager side effect\n",
        "class EagerLayer(tf.keras.layers.Layer):\n",
        "  def __init__(self, **kwargs):\n",
        "    super(EagerLayer, self).__init__(**kwargs)\n",
        "    # Do some kind of initialization here\n",
        "\n",
        "  def call(self, inputs):\n",
        "    print(\"\\nCurrently running eagerly\", str(datetime.now()))\n",
        "    return inputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5DFvc9ySr7t3"
      },
      "outputs": [],
      "source": [
        "# Create an override model to classify pictures, adding the custom layer\n",
        "class SequentialModel(tf.keras.Model):\n",
        "  def __init__(self):\n",
        "    super(SequentialModel, self).__init__()\n",
        "    self.flatten = tf.keras.layers.Flatten(input_shape=(28, 28))\n",
        "    self.dense_1 = tf.keras.layers.Dense(128, activation=\"relu\")\n",
        "    self.dropout = tf.keras.layers.Dropout(0.2)\n",
        "    self.dense_2 = tf.keras.layers.Dense(10)\n",
        "    self.eager = EagerLayer()\n",
        "\n",
        "  def call(self, x):\n",
        "    x = self.flatten(x)\n",
        "    x = self.dense_1(x)\n",
        "    x = self.dropout(x)\n",
        "    x = self.dense_2(x)\n",
        "    return self.eager(x)\n",
        "\n",
        "# Create an instance of this model\n",
        "model = SequentialModel()\n",
        "\n",
        "# Generate some nonsense pictures and labels\n",
        "input_data = tf.random.uniform([60, 28, 28])\n",
        "labels = tf.random.uniform([60])\n",
        "\n",
        "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U3-hcwmpI3Sv"
      },
      "source": [
        "First, compile the model without eager. Note that the model is not traced; despite its name, `compile` only sets up loss functions, optimization, and other training parameters."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w2GdwhB_KQlw"
      },
      "outputs": [],
      "source": [
        "model.compile(run_eagerly=False, loss=loss_fn)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WLMXk1uxKQ44"
      },
      "source": [
        "Now, call `fit` and see that the function is traced (twice) and then the eager effect never runs again."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VCoLlZDythZ8"
      },
      "outputs": [],
      "source": [
        "model.fit(input_data, labels, epochs=3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jOk6feLOK1pR"
      },
      "source": [
        "If you run even a single epoch in eager, however, you can see the eager side effect twice."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MGIYwrKpK06e"
      },
      "outputs": [],
      "source": [
        "print(\"Running eagerly\")\n",
        "# When compiling the model, set it to run eagerly\n",
        "model.compile(run_eagerly=True, loss=loss_fn)\n",
        "\n",
        "model.fit(input_data, labels, epochs=1)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qwq_cnc8Lwf8"
      },
      "source": [
        "### Using `run_functions_eagerly`\n",
        "You can also globally set everything to run eagerly.  Note that this only works if you re-trace; traced functions will remain traced and run as a graph."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oFSxRtcptYpe"
      },
      "outputs": [],
      "source": [
        "# Now, globally set everything to run eagerly\n",
        "tf.config.run_functions_eagerly(True)\n",
        "print(\"Run all functions eagerly.\")\n",
        "\n",
        "# First, trace the model, triggering the side effect\n",
        "polymorphic_function = tf.function(model)\n",
        "\n",
        "# It was traced...\n",
        "print(polymorphic_function.get_concrete_function(input_data))\n",
        "\n",
        "# But when you run the function again, the side effect happens (both times).\n",
        "result = polymorphic_function(input_data)\n",
        "result = polymorphic_function(input_data)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pD-AQxEhua4E"
      },
      "outputs": [],
      "source": [
        "# Don't forget to set it back when you are done\n",
        "tf.config.experimental_run_functions_eagerly(False)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sm0bNFp8PX53"
      },
      "source": [
        "# Tracing and performance\n",
        "\n",
        "Tracing costs some overhead. Although tracing small functions is quick, large models can take noticeable wall-clock time to trace.  This investment is usually quickly paid back with a  performance boost, but it's important to be aware that the first few epochs of any large model training can be slower due to tracing.\n",
        "\n",
        "No matter how large your model, you want to avoid tracing frequently. This [section of the tf.function guide](function.ipynb#when_to_retrace) discusses how to set input specifications and use tensor arguments to avoid retracing.  If you find you are getting unusually poor performance, it's good to check to see if you are retracing accidentally.\n",
        "\n",
        "You can add an eager-only side effect (such as printing a Python argument) so you can see when the function is being traced.  Here, you see extra retracing because new Python arguments always trigger retracing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jsGQ4GQAP2Ve"
      },
      "outputs": [],
      "source": [
        "# Use @tf.function decorator\n",
        "@tf.function\n",
        "def a_function_with_python_side_effect(x):\n",
        "  print(\"Tracing!\")  # This eager\n",
        "  return x * x + tf.constant(2)\n",
        "\n",
        "# This is traced the first time\n",
        "print(a_function_with_python_side_effect(tf.constant(2)))\n",
        "# The second time through, you won't see the side effect\n",
        "print(a_function_with_python_side_effect(tf.constant(3)))\n",
        "\n",
        "# This retraces each time the Python argument changes,\n",
        "# as a Python argument could be an epoch count or other\n",
        "# hyperparameter\n",
        "print(a_function_with_python_side_effect(2))\n",
        "print(a_function_with_python_side_effect(3))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D1kbr5ocpS6R"
      },
      "source": [
        "# Next steps\n",
        "\n",
        "You can read a more in-depth discussion at both the `tf.function` API reference page and at the [guide](./function.ipynb)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "intro_to_graphs.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
